Range imaging systems, also referred to as range cameras, range imaging cameras, or ranging devices, determine both distance and intensity for objects in a scene to be imaged. One type of range imaging camera is known as a time-of-flight range imaging camera because it utilizes the phase and intensity differences in reflected light to determine the range of objects from the camera.
One common type of time-of-flight range imaging cameras utilizes Amplitude Modulated Continuous Wave (“AMCW”) technology. As discussed in more detail below, range imaging cameras utilizing AMCW technology, typically use a light source to emit an intensity modulated illumination signal to illuminate a scene to be imaged. The signal is then reflected back to the range imaging camera by objects within the scene. Multiple images of the reflected light are then captured by a specialized device, such as a charge-coupled device (“CCD”) or a complementary metal oxide semiconductor (“CMOS”) image sensor integrated circuit (“IC”). A processor is then used to calculate the phase change of the intensity modulated light from these multiple images for points across the imaged scene by comparing to a global reference modulation signal. These phase changes are then analyzed to determine the distance between the objects and the range imaging camera, thereby enabling the determination of the size, shape, and location of the objects in the scene. Range imaging cameras are therefore very useful in many varied applications such as automotive safety and control systems, multimedia user interfaces, and image segmentation and surface profiling.
Effective determination of the distance between a range imaging camera and an object to be imaged depends on the phase difference between the emitted intensity modulated illumination signal and the received signal. Because the image sensor and the illumination are generally modulated with the same signal, as light travels away from a range imaging camera, that light begins to slip out of phase with the modulation of the image sensor. Consequently, the shorter the distance traveled by the light, the more in-phase the light will be. In other words, light reflected from objects close to the range imaging camera has an intensity modulation that is in-phase with the modulation of the range imaging camera, while light reflected from distant objects has an intensity modulation that is out of phase with the image sensor. While the same signal is commonly used to modulate both the illumination signal and the image sensor, different signals may be used in some situations to achieve similar results.
While there are many ways to implement typical range imaging cameras, FIG. 1 has been provided to show a simplified version of one such range imaging camera. As shown in FIG. 1, a typical AMCW range imaging camera 100 may include a signalling device 101, such as an LED, laser diode, or other light source, to emit an intensity modulated illumination signal towards a scene to be imaged and an image sensor 102 to capture images using the portions of the emitted signal that are reflected back to the system 100 from the objects within the scene. A processor 103 then compares the emitted signal with captured images to determine the intensity and the phase shift for each pixel in the captured image. By doing so, the processor is able to determine the distance objects in the scene are from the system 100. The system 100 may optionally contain an input device 104 to control the system 100, a display 105, and a memory 106. These devices may be chosen from among any of the known devices for performing such functions. There are many known ways of modulating the signalling device 101 and the image sensor 102. For example, the signalling device may simply be a light source that is cycled on and off very quickly, while the image sensor 102 may be modulated using a high speed shutter or by controlling the image sensor. The processor 103 may also include a signal generator, or a similar device, to assist in modulating the signalling device 101 and the image sensor 102. However, a signal generator may also be separately provided.
As with a digital photograph, the captured images will typically be formed by an array of pixels, with objects in the scene being effectively mapped onto the pixel array. Unlike a typical digital image, the pixels of a range imaging camera will record not only the illumination intensity (which may include the color) of the light reflected from an object in a scene, but also the phase of the illumination modulation envelope. Accordingly, each of the pixels in a captured image will measure a phase component that is based on the light reflected back to that pixel by objects in the scene. Similarly, the phase will change slightly between corresponding pixels in consecutively captured images due to the continuous nature of the AMCW intensity modulation. Furthermore, pixels that correspond to areas of the scene that do not have objects in them will not receive a reflected signal, but will only receive ambient light or light reflected from another source (such as the sun). Typical range imaging camera sensors include background or DC rejection circuitry that allows more efficient use of the sensor's dynamic range for capturing the modulated light; therefore, background light is effectively ignored by the sensor. The invention is more specifically directed to the image sensors used in range imaging cameras.